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ARE APPLIED TO EDUCATION. THERE ARE PROBLEMS IN THE ATTEMPT 
TO DEFINE AN EDUCATIONAL GOAL AND IN THE ANALYSIS OF 
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EXISTS TOWARD THE PROSPECT OF DEVELOPING A COMPREHENSIVE 
QUANTITATIVE MODEL OF THE AMERICAN EDUCATIONAL SYSTEM. THIS 
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To summarize our thesis briefly at the outset, we find that cost- 
benefit analysis encounters severe difficulties when one attempts to 
apply it to education. We believe that fruitful results are likely to 
. cxwcxjr. wc uu HOC oexieve that it is impossible or 

that there is no point in trying. Certainly every effort must be brought 
to bear on rationalization of the Nation's educational system simply 
because it is such an Important element of our society. It is a very 

expensive element too; any contribution that can be made to its effective- 
. ness or efficiency will pay handsome returns. 

The difficulties are not one but several. To overcome some of them 
require extensive research and development programs. Others will 
require considerable dialogue between educators and the general public 



about the purpose of education; an attempt to analyze the nationwide 
educational operation may have as one of its advantages the generation of 
such dialogue. Other difficulties arise from the diversity of control of 
education. The primary control is the local school board;' there are nearly 
24,000 such independent boards. State Departments of Education have a 
degree of control which varies widely among states. The federal government 



exercises one element of control of forbidding racial discrimination in any 
educational institution that receives federal funds. 



Presentef at the Washington Operations Research Council's Second Cost- 

Effectiveness Symposium, Marriott Twin Bridges Motor Hotel, Mar. 13-14, 1967 
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•The federal government is becoming a strong supporter of cost- 
benefit analysis in education because it is allocating substantial funds 
to all sectors of education. Administrators of the funds and Co ngress 



are detnanding hard information about the accomplishments of these pro- 
grams and estimates of what might be accomplished by alternative programs. 
A few State Departmo^fo 

F ^ are axso Deginning CO snow- signs of 



interest in quantitative management techniques in general and co st 
benefit analysis in particular. 



Goals of Education , where one tries to get beyond the easy majestic 
generalities in order to list goals to which he can attach measures, he 
finds confusion. For example, there is the goal of equal educational 
opportunity for all children together with the generally accepted prop- 
osition that racial isolation detracts from educational opportunity. 

In conflict with it is the goal of the neighborhood school closely in 
tune with the parents' desires for their children's education. Society 
is not yet of a mind as to whether it prefers desegregation to the 

neighborhood school or vice versa. That uncertainty leaves the analyst 
considerably frustrated. 

But analysts are persistent people and determined to analyze in 
any case. Opportunity is not education, some will say (skipping over 
the condemnation of millions of children to second class citizenship) . 
Education is preparation for the full and productive life. < Music, 
dancing, art, literature, lectures, sports, drama, political par ticipation, 
civic affairs, church work, and a job.) Why not measwe the fullness and 
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and productivity of life and then set up some relations showing how 
different aspects of schooling contribute eventually to those outcomes? 
Maybe sometime, but not this year or the next. 

Well, let's get down to fundamentals like readlno. —j 

thmetic. Those are what education is really concerned with and we 

cixxuga by means of achievement tests can we not? 

There do exist good instruments for measuring some educational outcomes 

of this kind but not for others that many edunators would consider 

equally or more important: judgment, social competence, analytical 

capability, personal orientation, creativity, conceptualization. When 

one examines the lives of particularly successful individuals, these 

hard-to-measure abilities usually appear to count for a great deal more 

than skill in reading, arithmetic, chemistry, latin, algebra, grammar, 
and the like. 



While we are on the subject of goals, we may note that some educators 
speak of these dlfficult-to-measure items as the main goals of schooling 
and the school subjects as simply devices for attaining those goals, ' 

Thus iatin is not really' studied for its own sake but to teach the 
logical structure of language, to enhance deeper understanding of the 
origins and foundations of English, and hence to improve the lucidity and 
precision of communication, etc. If there is any truth to such statements 
it should be easy to demonstrate that curriculum design is mainly witchcraft. 



Not a very promising basis for meaningful quantitative 



analysis. 
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In brief, education has a multiplicity of goals, none of which is 

dominant. It is not even clear what all the goals really are. Some acknowl 

edged goals are mutually inconsistent. Some of the most easily defended 

goals (associated with success in adult life) mvist be measured so far 

1" the future chat chey cannot be used for evaluation of today's educa- 
tional programs. 

J gtplexity of the Educational Proreo -.. The educational process has 

a degree of complexity quite beyond that of any other process we have 

been accustomed to analyzing. Even military confUct between infantrymen 

• is a simple process beside education. The reason is that the Infantrymen 

are carrying out explicit roles in a highly organized way to operate a 

specific weapons complex in a highly planned way. The military unit is 

supposed to be a machine and the people in it merely parts of the' machine. 

Of course it is not quite that way in practice hut the deviations are 

usually not so serious that they cannot be treated by relatively simple 
Statistical devices. 

In analysis of the educational process we encounter all the diffi- 
culties of social sclenceat their worst. Education is a vast panoply of 
human and procedural interactions which completely dominate the effects 
we would like to analyze. As a simple illustration, let us consider two 
widely employed methods of teaching a child to read: m one method the 

child learns complete words directly while in the other the child learns 
Individual letters first and then learns words as combinations of letters. 

Our school system has taught millions of children to read by both methods. 
Which is better? 
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Strangely enough, we don't know. When. we look at some data relevant 
to how fast or how well children learned to read, we find that the signal 
seems tc be totally submerged in the noise. Knowledgeable educators 
explain why. The method is really a small part of learning to read ana 



it often does not make a great deal of difference what method is used. 

Tf 2 teacher happens to have a personal preference for a bad method, then 
that teacher may well have more success with that method than with a 



good *method . If there is real empathy between the teacher and the child 
then the child is going to be a very successful reader ho matter what 



method is used. If the child has already learned to read from his mother, 
then any method used at school is going to look great -so far as his perfor 
mance is used to measure it. 



If the teacher is extremely able and commonly uses any of several 
methods and can diagnose the child's abilities well enough to select an. 
especially suitable method for that particular child, then that method is 

V 

going to show up well on two counts: not only was it especially matched 

to the child but the child had a fine teacher. It might have been a 
terrible method for most children or for most teachers. 

If the teacher is first an athletic coach and doesn't know much about 
what he is teaching nor how to teach it but is an enthusiastic motivator 
children, then the children will learn the material themselves and 
whatever you call his method will turn out very well in the data, you 
collect to evaluate the method. 
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If the child's parents take a serious Interest In his school work 
and keep up with what he Is doing at school and make sure he understands 
all the material being presented at school, then the methods used at 
the sdhool will appear to be very effective when judged in terms of this 
child's achievement. 

If the child's playmates or classmates generally feel that school- 
work is interesting or fun or Important or. must be accomplished for 
whatever reason, then the child will usually learn regardless of teachers 
and methods. A recent massive survey carried out by the Office of Educa- 
tion indicated that the educational inclination of a child's classmates 
. was even more closely associated with his educational achievement than 
the quality of his teachers. 

All these positive interactions have their negative counterparts and 
there are many others. Thus we must accer the fact that the educational 
process is beset by a complex array of interactions of all size s and 
shapes between teacher and children, teachers and parents, children and 
parents, teachei;Sand curricula, parents and curricula, children and 
curricula, school administrators and everything, everything this year with 
everything last vear and the year before that. These interactions frequently 
override the thing we are looking for. One negative interaction between a 
child and a teacher in an early grade dan conceivably throw the child for 
a loss from which he will not recover in a lifetime because it can 

4 

permanently type him as a C student in his own mind and in the minds of 
other teachers in the school whereas actually he could have been a B student. 
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Thus it is altogether possible that we may sometimes not be able 
to assess the impact of an educational device or method upon a child 
without knowing much of his past history including his significant 
interpersonal experiences. 

Th e Multiplicity of Federal Programs . The’ fact that elementary 
and secondary education are locally controlled in this Nation means 
that analysis of the system will certainly depend on efforts of the 

i 

federal government for many years to come. No local school district or 
State could justify the large research and development costs required 
before a tangible pay-off can be expected. The nature, of the federal 
piTogird.iiis is noti such s,s to invite optiroisnij however# 

In Fiscal Year 1966, the Office of Education administered 68 programs 
of financial aid ranging in size from less than a hundred thousand dollars 
to nearly one billion dollars. They influenced in some way about 2,500,000 
teachers, 54,000,000 students, 125,000 schools and colleges and reached 
into almost every town, hamlet, county and city in the United States. 

These programs were administered by five different bureaus, 50 State De- 
partments of education, and some 24,000 separate school districts, each one 
a distinct administrative unit with near total autonomy in its operations. 

To add to this general picture, there are over 40 agencies of the 
Federal Government that disburse money for educational activities, and it 
is estimated that they touch in some way an additional 14,000,000 persons. 
In fact, the 68,000,000 people being touched by federal assistance 
to education is close to the number of people working - which was about- 
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75,000,000 In 'December, 1966. When a nation allocates resources enough 
to help that mny people in their educational purf- jit, then an accounting 
of this stewardship surely seems in order. On the other hand, while the 
Office of Education is the most visible agency responsible for information 

I 

reporting and cost-benefit studies in education, it handles only about 
25% of federal expenditures in this area and provides only about 8% of 
the Nation's total budget for elementary and secondary education. That 
is, the federal programs are all merely adjuncts to our basic educational 
system. The federal task is therefore one of evaluating numerous small 
perturbations of a massive system which itself is not even slightly under- 
stood in any quantitative operational terms. 

Educational Information Gathering Systems . One of the distinguishing 
features of our nationwide educational information system is its voluntary 
nature. Almost all of the original data flow from the field to the Office 
of Education imder cooperative arrangements which recognize mutual interests. 
Excepting the Civil Rights Compliance Program, nearly all information .dis- 
tributed by the Office of Education is returned to the sources from which 
it has been collected in a form cooperatively designed to help educators 
plan and execute their programs. The traditional goals of the reporting 

f 

system are that it: 



1. Improve the quality of education, 

2. Support Congressional and legislative actions. 



Measure the extent of educational programs. 
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4. Minimize duplication of data collection and, 

5. Maximize consistency of data. 

You Will observe that the decision-making elements in a data report- 
ing system With these objectives are minimal. At best the decision-making 
is advisory rather than obligatory, and emphasizes dissemination rather 
than decision as its chief criterion of usefulness. Thud the data collec- 
tion system is not oriented to program structure, but rather to the his- 
toric local nature pf the educational process, emphasizing local control, 
•and local decisions; it is a program to supply background information to 
the educational community. „e must recall again that the decision process 
in education is distinctly local in nature and influenced by specific 
local concerns. Only when these local concerns are extended upward to 
Beet a downward flow of funds from the federal government is there a 

commingling of interest in information that will support a decision process 
at other than local levels. 

let us consider a specific illustration. Title 1 of the Elementary 
and Secondary Education Act allocated approximately $990,000,000 of ' 
federal funds for the support of "educationally disadvantaged" youths. 

This program adds to the traditional function of collecting and dlaseminat- 
ing information a need for informat-fnn 

.oj. an evaluative nature. This kind 
Of information cuts at the heart of the traditional process so far as data 

gathering and dissemination are concerned, for the information mu.t now 

reveal some answer to the preplexing puestion of "What has been accomplished 

by the $990,000,000 investment?" Thus the informational process must adjust 

to this new dimension of reporting and provide types of data not needed 
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prior to the federal 
Needless to say, 
this new dimension of 



government's support at local educational .operations, 
the educational community is not leaping to embrace 
reporting. Local educational agencies, which have 



always had almost total control of their educational systems, worry about 
the implications of federal control inherent in evaluational kinds of data 
Hhat actions are contemplated by the federal government as a consequence 
of evaluation? Data for operational analysis of federal educational pro- 



grams will probably come in slowly. 



I nformation Requirements for Program R„df .eMn ^ Program budgeting 
structures tend to be imposed from the top. Data come up from the bottom, 
and herein lies an unremitting difficulty. A Program Planning and Budget- 
ing System, which broke out education as a major category, was recently 
devised by the Department of Health. Education and Welfare. It identified 
the following classification for which federal funds might be identified: 
First, there is an identification of the ultimate purpose of the program--, 
such as the development of basic skills and attitudes or vocational and 
occupational skills. The next set of Information seeks to refine these 
ultimate purposes by identifying whether the resources are to extend 
present programs or. to introduce new ones. The next element of informa- 
tion identifies the Impact levels— such as pre-schbol. elementary 
secondary, and so on. The final data element is the identification of 
the program recipient-such as an Individual grant, an institutional 
grant, a local agency grant, and so on. The structure permits an easy 
identification of how much is spent by purpose, by educational level, and 
by ultimate program receipient. such as an individual, a state agency, or 
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a local agency. It is mainly an operational accounting system by means 
of which one can identify program costs. 

Some such system as this is absolutely essential to meaningful cost- 
benefit analysis of our nationwide educational system. We cannot do cost 
analysis without costs. Implementation of such, a system will however, 
require revolutionary revision of the accounting systems of 25,000 school 
districts in the long run. These districts are, of course, accustomed 
to setting up their own accounts to suit their own requirements. They 
may well concede that program accounting has something to recommend it, 
but we can be sure that most of them will resist such a major revision 

on grounds that the cost will exceed what benefits they can visualize it • 
bringing to their own operations. 

M easurement Problems . We do not know what education costs. Estimates 
have been made, and they range up to $50,000,000,000 per year, but these 
are money costs alone and reflect a lower bound only. Few school districts 
and perhaps no colleges allow for depreciation of equipment. Few educa- 
tional activities allocate costs that are joint in nature. Fewer still 
capitalize the inv^-%*jment cost needed to erect and modernize new facili- 
ties. What is needed, before any real benefit-cost studies can be made 
of education, is an entirely new data collection system and a set of agree- 
ments on what the "real" costs of education are. Should costs borne by 
individual students be included? What about their opportunity costs? 

But costs are easy relative to benefits. Even the financial bene- 
fit is most elusive and if there is anything we. analysts ought to be 
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expert at it should be calculation of that kind of benefit. Much has been 
said about the increased income a person derives from Increased education 
but the essential question is still "How much more productive has he be- 
come because of the marginal dollar spent on his education?" Efforts to 
answer this question in terms of real data on income versus education en- 
counter the following formidable hurdle: High incomes tend to go 'to able 
people. Able people tend to do well in school and hence get a substantial 
amount of education. Also they tend to have able parents -whose economic 
success tends to assist their children up the economic ladder in various 
ways. Thus only a fraction of the added Income of educated people can 
be attributed to their added education. A fairly elaborate model and an . 

extensive longitudinal data collection program will be required to estimate 
what that fraction is. 

We have already noted that tests exist for measuring the achievement 
of children in certain academic disciplines. But even here the data may 
not be immediately forthcoming because of political problems. There is 
a major development program under way to establish a nationwide periodic 
assessment of educational achievement; it is being done under private 
auspices with generous financial support from the Carnegie Corporation 
and the Ford Foundation. The director of the development program is a 
very distinguished educator, Ralph Tyler, Head of the Center for Advanced 
Study in the Behavioral Sciences. It sounds promising but it is far from 
a sure thing. A sizable section of the educational community has taken 
a strong position against it. Recently the executive secretary of the 
national organization of school superintendents circulated his membership 
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urging total resistance to this proposed assessment program and refusal 
of access to Dr. Tyler to any school for the puirpose of trying out his 
new assessment instruments. 

We have already noted that no instruments at all exist for measuring 
some of the most important outcomes of education. Some beginnings have 
been made for creativity and also for personal orientation, but others 
have not even been thought about. Some of these areas would find great 
practical difficulty even if instruments existed because they would involve 
to some degree an invasion of privacy. One cannot go poking into, for 
example, family psychological problems even though they might have a 
very strong bearing on the kind of education appropriate for the child. 

In our discussion of the many significant interactions that occur 
in the educational process, it became evident that we shall never con- 
struct an adequate quantitative description of the educational process 
without measures of these interactions and extensive data regarding their 
statistical distribution through the school system. Thus we are faced 
with the prospect of developing a whole new class of measuring instruments 
which have to do, among other things, with the difficult matter of the 
degree of rapport between two humans. 

Outlook . We are not pessimistic about operations analysis of the 
American educational system. We are merely convinced that a comprehensive 
quantitative model is a very big job which will require large resources 
and many years. We have explored these difficulties to indicate why the 
job is big and to show what some of the crucial elements are. 
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Certainly rudimentary analyses can be done now and in the near future. 
We are doing such things now. A survey of educational opportunity done by 
the Office of Education last year reached 600,000 pupils and that was a 
large enough sample that a few signals could be detected in the noise. 

And there is reason for optimism with respect to the whole problem. 
There is widespread recognition that analysis of the system is mandatory. 
We cannot allocate over $50 billion per year to an activity without making 
substantial efforts to allocate it effectively. The federal government 
is now committed to such analysis; it has created a division containing 
twenty professional people in the National Center for Educational Statis- 
tics of the U.S. Office of Education for just the pu^ose of developing 
models of the American educational system. 

More Importantly, there is now real promise of effective research 
of educational processes. In the past, educational research has been 
done mainly in small projects which could be financed by university de- 
partments of education. Mostly they revolved around Ph.D, theses. That 
kind of attack could not possibly illuminate such a complex operation as 
education. Recently the federal government has begun to allocate some 
$100 million per year to educational research. Much of that money is 
going to several newly created multidisciplinary research institutes 
which will be able to attack some of the problems we have discussed on 
the scale required. In a few years we should begin to get an understand- 
ing of all those Interactions that occur in the educational process, their 
relative sizes, their relationships, and what can be done to modify them. 

With that kind of knowledge, we can eventually develop a satisfactory 
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quantitative model of the operation. Then we shall have a context within 
which meaningful information can be sought about matters under the direct 
control of school administrators: curricula, methods of teaching, school 

organization, class size, teacher training, tracking, integration, archi- 
tecture, equipment, personnel policies, diagnosis of childrens* abilities, 
matching of teachers and methods and curricula to ctiildren, etc. ' Some 
day a school superintendent will be able to say something like this to 
his community: "With the total budget yoti propose 1 .can get an annual 

average of about 31% of your high school g;raduates d.nto college; with 
another $100 per pupil I could get it up to 44% and with still another 
$100 I could raise it to 54%. " 
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